Method and apparatus for storing and retrieving data embedded into the surface of a 3d printed object

ABSTRACT

A computer implemented method and apparatus for storing and retrieving data embedded into the surface of a 3D printed object is described. The method and apparatus develops an electronic file used for printing a 3D object which embeds as structure into the 3D object, a 3D symbol matrix representative of data to be printed concurrently with the 3D object, such as a 3D barcode. A selected symbology is used for making the symbol matrix in accordance with the type of printing process to be used to print the 3D object.

BACKGROUND OF THE INVENTION

1. Field of the Invention

This disclosure relates to 3-dimensional (3D) printing of objects and,more particularly, to techniques for storing and retrieving dataembedded into the surface of a 3D printed object.

2. Description of the Related Art

Technology currently available and well known provides for 3-dimensional(3D) printing of physical 3D objects. Such objects can be the shape ofany physical creation conceived by man, from something as simple as ageometric shape like a ball or cube to something as complex as a replicaof an intricate trestle bridge, a statue by the artist Michelangelo, ora representation of an exotic animal. As is typical of creations thatresult from human effort, it is desirable to associate information withsuch creations, such as the name of the entity, person or company thatcreated the object, the name of entity, person or company, that printedthe object, a creation date, a name for the object, or any otherinformation that entity would like to have associated with the 3Dobject. It is typically desired that such information be as permanentlyassociated with the object as possible. One way humans conveyinformation is by writing. Written information communicates facts (alsocalled data) that are acquired at a given point in time and thenconveyed to others by an arrangement of symbols. Symbols can include,for example, numeric characters such as 1, 2, 3, . . . etc., alphabeticcharacters such as A, B, C . . . etc., which examples are representativeof Arabic numerals and Latin letters. Many other kinds of letters andnumeric characters are also known and used that are based on differentlanguages, such as Chinese, to name just one other language.

One technique used to associate written information so that it ispresented in combination with a physical object, is to convey theinformation by use a label having written characters printed thereon.One type of label designed to function particularly well in a digitalenvironment is a barcode. A barcode is a type of label which is easy fora machine to create and simple for a machine to reliably read. Abarcode, like any other label, is an arrangement of symbols that conveysinformation, such as an identification, to be associated with a physicalobject. Although the identification of an object can be conveyed by alengthy description of the object, which will require the use of manysymbols, a barcode may be a unique and predefined arrangement of symbolsthat can be used to convey the identification information with lesssymbols. Originally, barcodes used spaced parallel lines as symbols,which systematically represent data by varying the widths and spacingsof the parallel line symbols. Such barcodes are referred to as linear orone-dimensional (1D) since their varying widths and spacings can bedetected by moving an edge detector in a single direction (i.e., 1D) soas to traverse the parallel lines. In the last decade 1D barcodes haveevolved into rectangles, dots, hexagons and other geometric patternsarranged in two dimensions (2D). By having two dimensions to presentdata, more data can be conveyed with a 2D barcode as compared with a 1Dbarcode. For example, one kind of object identification information thatcan be in a barcode is called a stock-keeping unit (SKU) number. An SKUnumber is a unique identifier that is assigned by a business for eachdistinct product that can be purchased from that business, such asSKU#123456789. Thus, detailed identification information for a physicalobject can be compactly conveyed by merely providing a barcode that hasthe SKU number for that physical object.

In order that the barcode identification information (such as an SKUnumber or other arrangement of information conveying symbols) does notbecome separated from the physical object, barcodes are typicallyprinted on a label and then attached to the physical object, such as byglue or other adhesive.

Other types of information can be included in a barcode, such as anExpiration date, an Owner, a Universal Resource Locator (URL), a Batchnumber, a Watermark, a Production date, a Package date, a Serial number,a Weight (kgs/pounds), a Price, a Universal Product Code (UPC), aRevision Number, a Phone Number, or any other information imaginable.The only limit on the amount of information that can be conveyed by abarcode type label is the amount of space on the label in which to writethe symbols used to convey the information. Since the size of the labelis limited by the size of the product on which it is to be placed, it isalways desirable to find a way to improve the amount of information thata label of a limited size, such as a barcode, can carry. Currently onedimensional (1D) and two dimensional (2D) barcodes are ubiquitouselements in modern manufacturing processes and products. Barcodes basedon Universal Product Code (UPC) and the “GS1” standards are numeric dataused to identify products and allow businesses to track their productsfrom creation to sale.

Most current 1D and 2D barcodes are limited to printing 2Drepresentations directly on an object or on a label and then adheringthe label to a surface of the object. Sometimes the 2D representationscan be micro-etched on hard surfaces of the object. As previously noted,2D barcodes provide more information as compared with a 1D barcodebecause two dimensions of the label are available for writing thesymbols that convey the data as compared with one dimension. However,even when 2D surface barcodes are used, the amount of data that can beconveyed is limited by the 2D area of the surface used on which thesymbols are written. Additionally, the durability of the data presentedby a 2D barcode is low because a surface presentation, such as a label,can be easily damaged (for example by the action of a liquid, such aswater, degrading the ink used to print the barcode, or by waterdegrading the adhesive used to affix the barcode to the surface of theobject). Even a 2D barcode presentation that is etched into the surfaceof an object can be damaged to the point of being partially obscured bybeing worn away by repeated handling of the object or by exposure of theobject to the forces of nature over time. Even furthermore, cryptologyand steganography techniques for providing robust data security areunnecessarily limited when using 2D barcodes, since they are limited totechniques that only use two dimensions.

SUMMARY OF THE INVENTION

The Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used as an aid in determining the scope of the claimed subjectmatter.

A method and apparatus for storing and retrieving data embedded in thesurface of a 3D printed object is described. The method and apparatusdevelops an electronic file used for printing a 3D object which embeds a3D symbol matrix as part of the surface structure of the 3D object. The3D symbol matrix is representative of data, and is printed concurrentlywith printing of the 3D object. In some embodiments, a selectedsymbology is used for making the symbol matrix. The symbology used isselected so as to optimize accurate symbol printing, by matching thesymbology with the accuracy level achievable by the type of printingprocess used to print the 3D object.

A method and apparatus to retrieve data embedded in the surfacestructure of a 3D object, where a 2D image of data embedded in thesurface structure of the 3D object is processed to:

-   -   a) identify an orientation and dimension of a bounded area of        the 3D object that includes the data,    -   b) identify sub-regions of the area, and    -   c) detect symbols in the sub-regions of the area so as to        retrieve the data.

A printed 3D object comprising a surface having embedded therein duringprinting, a volume corresponding to a 3D symbol matrix representative ofdata.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 depicts a block diagram of a system for storing data embedded inthe surface structure of a 3D printed object and retrieving data storedin the surface structure of a 3D printed object, according to one ormore embodiments;

FIG. 2 depicts a flow diagram of a method for storing data embedded inthe surface structure of a 3D printed object as performed by the dataencoding processor and symbol embedding module of FIG. 1, according toone or more embodiments;

FIG. 3 depicts a flow diagram of a method for storing data embedded inthe surface structure of a 3D printed object as performed by the dataencoding processor and symbol embedding module of FIG. 1 for a firsttype of printing process, according to one or more embodiments;

FIG. 4 depicts a flow diagram of a method for storing data embedded inthe surface structure of a 3D printed object as performed by the dataencoding processor and symbol embedding module of FIG. 1 for a secondtype of printing process, according to one or more embodiments;

FIG. 5 depicts a flow diagram of a method for retrieving data embeddedin a surface structure of a 3D object as performed by the imageprocessing module and the data decoding and error correction module ofFIG. 1, according to one or more embodiments;

FIGS. 6A and 6B together depict construction of a symbol matrix asperformed by the symbol embedding module of FIG. 1, according to one ormore embodiments, and

FIG. 7 depicts embedding a symbol matrix in a 3D object as performed bythe method of FIG. 4, according to one or more embodiments.

While the method and apparatus is described herein by way of example forseveral embodiments and illustrative drawings, those skilled in the artwill recognize that the method and apparatus for storing and retrievingdata embedded in the surface structure of a 3D printed object is notlimited to the embodiments or drawings described. It should beunderstood, that the drawings and detailed description thereto are notintended to limit embodiments to the particular form disclosed. Rather,the intention is to cover all modifications, equivalents andalternatives falling within the spirit and scope of the method andapparatus for storing and retrieving data embedded in the surfacestructure of a 3D printed object defined by the appended claims. Anyheadings used herein are for organizational purposes only and are notmeant to limit the scope of the description or the claims. As usedherein, the word “may” is used in a permissive sense (i.e., meaninghaving the potential to), rather than the mandatory sense (i.e., meaningmust). Similarly, the words “include”, “including”, and “includes” meanincluding, but not limited to.

DETAILED DESCRIPTION OF EMBODIMENTS

As previously explained, existing barcode solutions provide 2Drepresentations on surfaces of objects as barcodes. However, when 2Dsurface barcodes are used, the amount of data that can be conveyed islimited by the 2D area of the barcode used to form the symbols.Additionally, the durability of their data presentation is low becauseassociating the data of a barcode to an object by use of a surface labelcan be easily damaged (for example by water degrading ink used to printthe barcode or by water degrading the adhesive used to affix the barcodeto the surface of the product) or the printed barcode may worn away overtime. Even furthermore, cryptology and steganography techniques forproviding robust data security are unnecessarily limited, since they arelimited to techniques that only use two dimensions.

Thus, and in accordance with embodiments of the present invention,techniques are provided herein for the creation, association, andretrieval of data that: (1) facilitates data presentation in a 3D mannerthat is improved as compared with data presentation in a 2D manner, (2)a durability of data presentation that is improved as compared with asurface label (either printed or surface etched) for data presentationand (3) a data presentation having data security that is improved ascompared with the data security that is possible with 2D datapresentation. For example, if the data to be associated with a 3D objectand conveyed by writing is a barcode such as SKU#123456 (although anyother type of data imaginable by man is possible to be conveyed, andbarcode data is only one example of billions of possibilities of datathat can be conveyed by writing), instead of providing a 2D barcode onthe surface of the 3D object to convey data, embodiments of theinvention provide a 3D barcode that is embedded into the surfacestructure of the 3D object. The embodiments provide that the 3D barcodeincludes symbols representative of data, which symbols become embeddedinto a volume of the 3D object that extends into the 3D object from anoutside surface thereof, by modifying the printing process of the 3Dobject to include printing of the symbols that comprise the informationconveyed by the 3D barcode. That is, the modification of the 3D printingprocess causes the symbols of the 3D barcode to structurally become partof a volume of the 3D object during printing of the 3D object. Thus, the3D barcode becomes formed from the same material and at the same time asthe 3D object is formed by printing. By embedding the 3D barcode symbolsinto a volume of the 3D object that extends into the surface of the 3Dobject, the durability of the barcode information is greatly enhanced ascompared with a printed paper label that is merely adhered to thesurface of the 3D object or a surface etching of a 2D barcode onto asurface of the 3D object. Also, by use of a volume in which to form thesymbols, a higher density of data can be presented by use of the thirddimension in which to include the symbols. Even furthermore, datasecurity (using either of cryptographic or steganographic techniques) issubstantially improved as compared with 2D techniques, since when athird dimension is available for cryptographic or steganographictechniques, more complex and therefore successful techniques can beimplemented.

Embodiments of the present invention include techniques for storing andretrieving data embedded in the surface structure of a 3D printedobject. The embodiments develop an electronic file used for printing a3D object that will store the data by embedding it into a volume of a 3Dobject from an outside surface thereof, during printing of the 3Dobject. The data is structured as a matrix of 3D symbols (such as anarrangement of objects having a simple geometric shape) which symbolsare representative of the data to be printed concurrently with the 3Dobject.

In some embodiments, the symbology used for making the symbol matrix isselected in order to optimize accurate symbol printing by matching thesymbology with the accuracy level achievable by the type of printingprocess used to print the 3D object. For example, a Fused DepositionModel (FDM) type printer is particularly suitable to accurately print anarrangement of parallel lines, while a Selective Laser Sintering (SLS)type printer is more suitable to accurately print an arrangement ofsolid geometric shapes. The symbology used to represent the data and thestructure of the 3D symbol matrix are also selected so as to allowsimple image processing techniques to be applied to a 2D image of the 3Dsymbol matrix, so as to provide an easy and reliable identification ofthe symbols representative of the embedded data. For example, “edgedetection” that is, detection of edges, is a simple and reliable imageprocessing technique where brightness variations from light to dark, orvice versa, are detected in areas of an image. Each brightnessvariation, and its position in the image, is recorded as an edge. Incombination, once all edges and their positions are detected, objectshaving a simple shape in the image (such as a group of parallel lines orthe hypotenuse of a right triangle) are reliably, simply and directlyindicated by the pattern of the detected edges. Accordingly, thesymbology used in the matrix to convey the data includes simplegeometric shapes or lines, so as to allow simple and reliable recoveryof the data embedded in the 3D object by the 3D symbol matrix usingsimple 2D image processing.

As with any symbology used as characters of a language to communicateinformation, each character of the language has a meaning. In oneembodiment each of a plurality of different symbols can be pre-definedby the language to convey a given number. For example, if the symbologyincluded four solid right isosceles triangles, each one orientated so asto be rotated 90 degrees around a central vertical axis from oneanother, each particular orientation of the four triangles can be usedto convey a respective one of four numeric values, such as 0, 1, 2 and3. Alternatively, if the symbology included four different orientationsof rows of parallel ridges, each ridge symbol orientated so that therows of parallel ridges are rotated 45 degrees with respect to the rowsof each other ridge symbol, each particular orientation of the fourridge symbols can be used to convey a respective one of four numericvalues, such as 0, 1, 2 and 3 (for rotations of +45 degrees fromhorizontal, −45 degrees from horizontal, +90 degrees from horizontal,and −90 degrees from horizontal). With such symbols and pre-definitionof what they mean (that is, how to decode them so as to conveyidentification of each symbol to identification of a numeric value thatcorresponds, via mapping, to a character in a human communicablelanguage), conveyance of information to humans is possible using thesymbols.

As noted above, the embodiments described herein make use of the thirddimension of a 3D volume as compared with using only a 2D area, toinclude data with a 3D object in a manner that aids steganography, highdata storage capacity, durability, and data security (protection frommalicious intent). Obtaining the data to be included with the 3D objectcan come any time before the actual printing of the 3D object. The 3Dobject may be obtained with the data already included therein, or the 3Dobject may be obtained separately from the data and then included withthe 3D object just before the 3D printing. In general, however,obtaining the data can be done anytime during or after creation of theelectronic file representative of the 3D object that is used forprinting of the 3D object.

Embodiments of the invention may be realized as a workflow in a 3Dprinting program, such as PHOTOSHOP 3D™, an image viewing and editingcomputer software application available from ADOBE® SystemsIncorporated. PHOTOSHOP 3D™ allows a direct connection between a users'computer running PHOTOSHOP 3D™ and a 3D Printer (that is, without usingany intermediate printing software) for printing 3D objects. Assemblingdata obtained for association with a 3D object into a 3D barcode forprinting simultaneously with the 3D object, can be done within thePHOTOSHOP 3D™ software automatically, when the electronic filerepresentative of the 3D object already includes the data, or manually,by the user merely typing the data on a keyboard during operation of thePHOTOSHOP 3D™ software on the users' computer. In another embodiment,the workflows described herein can be incorporated into 3D printingsoftware that is included with the purchase of a 3D printer, or can beused by a Print Service Provider (PSP) such as Shapeways, which prints3D objects for users who send their electronic files to them forprinting.

Exemplary uses of the invention are: 1) Consumer appliance companieswhich create 3D prints of various iterations of their products, and inorder to not lose track of which model is which, they incorporate the 3Dprint date, source model ID, revision number, etc. onto the 3D objects,so it will forever be part of the structure of the product; 2) Designerswho create 3D objects and make them available for 3D printing toconsumers in a 3D printing marketplace, such as at Shapeways, typicallywant that whenever their model is printed, a URL to their website willalso be incorporated into the 3D print of their 3D object. Optionally abacktrack URI (Uniform Resource Identifier) should be included with the3D object, so as to will allow the designer to uniquely track feedbackby the purchasers of each 3D object; 3) Industrial production lines mayuse 3D printing for producing parts for their final products. As such,it would be desirable if each part included its own ID number,batchCode, Manufacture Date, Ship to Customer details, etc., directlyand permanently therein; 4) Companies may create 3D prints of their newproducts for sharing with close internal partners, but may not wanttheir designs to be disclosed to others, and if such a case happens theywould like to be able to backtrack to the source of any unwanteddisclosure. Hence as a matter of steganography, ID data added into the3D print in a manner so that it is not overtly visible in the 3D print.Thus, by use of the present invention, a Partner Identifier can besurreptitiously inserted into their 3D prints.

Embodiments of the invention find use in industrial applications,enterprise applications and with end users. Any workflow where it isdesirable that more than a token amount of information be included witha 3D object for the purpose of steganography, data storage capacity,durability or data security.

Various embodiments of techniques for storing and retrieving dataembedded in the surface structure of a 3D printed object are described.In the following detailed description, numerous specific details are setforth to provide a thorough understanding of claimed subject matter.However, it will be understood by those skilled in the art that claimedsubject matter may be practiced without these specific details. In otherinstances, methods, apparatuses or systems that would be known by one ofordinary skill have not been described in detail so as not to obscureclaimed subject matter.

Some portions of the detailed description that follow are presented interms of algorithms or symbolic representations of operations on binarydigital signals stored within a memory of a specific apparatus orspecial purpose computing device or platform. In the context of thisparticular specification, the term specific apparatus or the likeincludes a general-purpose computer once it is programmed to performparticular functions pursuant to instructions from program software.Algorithmic descriptions or symbolic representations are examples oftechniques used by those of ordinary skill in the signal processing orrelated arts to convey the substance of their work to others skilled inthe art. An algorithm is here, and is generally, considered to be aself-consistent sequence of operations or similar signal processingleading to a desired result. In this context, operations or processinginvolve physical manipulation of physical quantities. Typically,although not necessarily, such quantities may take the form ofelectrical or magnetic signals capable of being stored, transferred,combined, compared or otherwise manipulated. It has proven convenient attimes, principally for reasons of common usage, to refer to such signalsas bits, data, values, elements, symbols, characters, terms, numbers,numerals or the like. It should be understood, however, that all ofthese or similar terms are to be associated with appropriate physicalquantities and are merely convenient labels. Unless specifically statedotherwise, as apparent from the following discussion, it is appreciatedthat throughout this specification discussions utilizing terms such as“processing,” “computing,” “calculating,” “determining” or the likerefer to actions or processes of a specific apparatus, such as a specialpurpose computer or a similar special purpose electronic computingdevice. In the context of this specification, therefore, a specialpurpose computer or a similar special purpose electronic computingdevice is capable of manipulating or transforming signals, typicallyrepresented as physical electronic or magnetic quantities withinmemories, registers, or other information storage devices, transmissiondevices, or display devices of the special purpose computer or similarspecial purpose electronic computing device.

FIG. 1 is a block diagram of an apparatus 100 for storing data embeddedin the surface structure of a 3D printed object and retrieving datastored in the surface structure of a 3D printed object, according to oneor more embodiments. The apparatus 100 includes a computer 102. Thecomputer 102 is a computing device, for example a desktop computer,laptop, tablet computer, and the like. The computer 102 includes aCentral Processing Unit (CPU) 104, support circuits 106, and a memory108. The computer 102 may be connected to a 3D printer 110. The CPU 104may include one or more commercially available microprocessors ormicrocontrollers that facilitate data processing and storage. Thevarious support circuits 106 facilitate the operation of the CPU 104 andinclude one or more clock circuits, power supplies, cache, input/outputcircuits, and the like. The memory 108 includes at least one of ReadOnly Memory (ROM), Random Access Memory (RAM), disk drive storage,optical storage, removable storage and/or the like.

The memory 108 includes an operating system 112, a data encodingprocessor 114, a symbol embedding module 116, data 118 to be encodedinto a 3D object 120, a 2D image 122, an image processing module 124 anda data decoding and error correction module 126. It is noted that data118 is any information that is to be conveyed in association with the 3Dobject using written symbols, such as characters, letters, numbers,pictographs, etc., or any combination thereof. As well known, charactersare grouped by coding (mapping) into what is called “language” so as toconvey information, such as accomplished by the alphabetic charactersused to convey the English language, and the characters used to conveythe Chinese language, to name just two of many possible charactercodlings used to convey information by language and may include data118. Thus, data 118 may be “www.google.com”. Other groupings ofcharacters used to convey information are numeric based languages, suchas ASCI, Binary, Hexadecimal, etc., which upon application of apre-defined mapping, is converted into non-numeric language, such asEnglish. Thus, data 118 may be a combination of letters and numbers,such as “SKU#12345”. The characters used to form language and conveyinformation, and particularly numeric characters, may also be easilyencoded, that is re-mapped in accordance with a pre-defined mapping, soas to increase the amount of information conveyed, or to increase theability of the data to be completely understood when parts of the dataare unrecoverable or erroneously recovered due to errors, or to obscurethe presentation of the underlying data. In some embodiments describedherein the data 118 is encoded for all three of the above-noted reasons,as will be described below.

The 3D object 120 is an electronic file representative of a 3D object tobe printed, such as a conventional STL file. The 2D image 122 isprovided by an image source 123. Image source 123 is any digital imagingdevice that is readily available to someone who may have a 3D printedobject in their presence and who is willing to take an image of the 3Dobject so as to recover the data 118. Image source 123 need merelyprovide a digital image of conventional photographic quality of aportion of the surface of the 3D object which includes data 118 printedtherein. As such, image source 123 may be a camera of a cell phone, notspecifically shown. Of course, if a person in the presence of the 3Dobject has a stand-alone digital camera, that cameral could also be usedto supply an image of the 3D object. The operating system 112 mayinclude various commercially known operating systems.

The data encoding processor 114 accesses the data 118 to be stored inthe 3D object 120. As noted above, the data 118 may have been includedwith the electronic file representative of the 3D object 120, or mayhave been provided for association with the 3D object after creation ofthe 3D object, by manual input to memory 108 by the user of computer102. The data encoding processor 114 then converts the data 118 to codeddata, that is, code words 115 representative of the data 118. Asdescribed above, converting data into coded data provides advantages,such as the ability to increase the amount of information conveyed bythe data, increasing the recovery of the data in view of errors, etc. Inorder to further increase the durability and reliability of correctrecovery of the code words 115 that are to be embedded in the 3D object120, forward error correction coding is optionally employed as part ofthe coding process. One suitable known forward error correction codingtechnique is the well-known and widely used Reed-Solomon ErrorCorrection Code (RS-ECC) technique, described in greater detail withrespect to the method 200 below. The code words 115 are then applied tothe symbol embedding module 116. In accordance with embodiments of theinvention, symbol embedding module 116 operates differently dependingupon which printing technology is to be used to print the 3D object 120.If a linear extrusion/tracing type of printing process is to be used,such as by a Fused Deposition Model (FDM) printer, the symbol embeddingmodule 116 operates in accordance with method 300 of FIG. 3. If agrid-based layered area type of printing process is to be used to printthe 3D object 120, such as by Digital Light Projection (DLP), PowderDeposition Printing (PDP), Jetting Liquid Polymer (JLP), Selective LaserSintering (SLS) or Stereolithography (SLA) printer, the symbol embeddingmodule 116 operates in accordance with method 400 of FIG. 4. The outputof the symbol embedding module 116 is a set of printing instructions 117(an electronic file) adapted to a particular printing technology formodifying the printing of 3D object 120. When the set of printinginstructions 117 is applied to a printer 110 of the appropriatetechnology, it causes the printer to print the 3D object 120 in a mannerso as to have the data 118 embedded into a surface area of the 3D objectas symbols that form part of the structure of the 3D object.

The end result when using a grid-based area type of printing process isa 3D object such as 702 of FIG. 7, having a data matrix 706 embedded asstructure in a surface area thereof, such as an area on a bottom surface708. The embodiments rely on the reflective properties of differentlyshaped symbols formed by the material in the embedded data matrix 706 torecover the data 118.

For recovery of the data embedded into a surface area of the 3D object,the illustrated embodiment includes use of the image processing module124 and the data decoding and error correction module 126. The 2D image122 (embodied as an electronic file, such as a JPEG) generated by theimage source 123, is provided by support circuits 106 to the imageprocessing module 124. The image 122 is processed by the imageprocessing module 124 using conventional edge detection techniques wellknown to those of ordinary skill in the art, so as to detect patterns inlight reflected from the symbols embedded as structure into the surfacearea of the 3D object. Initially, the image 122 is processed by imageprocessing module 124 so as to identify the area of the surface wherethe data is embedded into the 3D object. This is accomplished relativelyeasily by the use of a predefined pattern of the embedded symbols thatis arranged to define the bounds and orientation of the area whichincludes the data. Image processing module 124 uses edge detectiontechniques to detect patterns in light reflected from the symbols, asdescribed below in conjunction with FIG. 5. Additionally, anotherpredefined pattern of the embedded symbols is used to identify anddefine a grid pattern within the identified area in which the symbolsrepresentative of the data are located. Additional details concerningdefining and identifying the data matrix and the symbols includedtherein are also described below in conjunction with FIGS. 6 and 7. Oncethe grid pattern in the area where the data is located is identified,image processing module 124 processes the image 122 in the identifiedgrid pattern to identify the symbols representative of the data. Theidentified symbols are then processed by the data decoding and errorcorrection module 126, which operates in a well know manner inversely tothe above-described Reed-Solomon Error Correction Coding technique usedby the data encoding processor 114, for recovering the error correctioncodes and then decoding them so as to recover the original data.

The above described operations can be realized as a workflow operatingin a 3D printing program, such as PHOTOSHOP 3D™, an image viewing andediting computer software application available from ADOBE® SystemsIncorporated, which program can be installed in memory 108. PHOTOSHOP3D™ allows a direct connection between a users' computer runningPHOTOSHOP 3D™ and a 3D Printer (that is, between computer 102 andprinter 110) without using any intermediate printing software, forprinting 3D objects. Assembling data obtained for association with a 3Dobject into a 3D barcode for printing simultaneously with the 3D object,can be done within the PHOTOSHOP 3D™ software automatically, if theelectronic file representative of the 3D object already includes thedata, or manually if the data is not included with the electronic file,by the user merely typing the data on a keyboard (not shown) duringoperation of the PHOTOSHOP 3D™ software on the users' computer 102. Inanother embodiment, the workflows described herein can be incorporatedinto 3D printing software that is included with the purchase of a 3Dprinter, or can be used by a Print Service Provider (PSP) such asShapeways, which prints 3D objects for users who send their electronicfiles to them for printing.

Although in FIG. 1 the data recovery modules, namely the imageprocessing module 124 and the data decoding and error correction module126, are illustrated as part of computer 102, in other embodiments, datarecovery can be accomplished just as easily at another location, using aprocessor in a computer other than computer 102.

FIG. 2 depicts a flow diagram of a method 200 for storing data assymbols embedded into the surface of a 3D printed object as performed bythe data encoding processor 114 and symbol embedding module 116 of FIG.1, according to one or more embodiments. The method 200 encodes the datainto symbols that are adapted to a particular printing technology to beused to print the 3D object. The method 200 then develops a set ofinstructions, that is, and electronic file, for printing the 3D objectso that the data becomes stored as a matrix of symbols (such as symbols620-626 of the matrix 600 depicted in FIG. 6A or symbols 632-638 of thematrix 706 depicted in FIGS. 6B and 7) embedded into the surfacestructure of the 3D object upon printing. Method 300 of FIG. 3 uses asymbology such as shown in FIG. 6A, which is optimized to achieveaccurate symbol printing for a linear extrusion/tracing type of printingprocess, such as for a Fused Deposition Model (FDM) printer. Method 400of FIG. 4 uses a symbology such as shown in FIG. 6B, which is optimizedso as to achieve accurate symbol printing for a grid-based sequentiallayered area type of printing process, such as for a Digital LightProjection (DLP), Powder Deposition Printing (PDP), Jetting LiquidPolymer (JLP), Selective Laser Sintering (SLS) or Stereolithography(SLA) printer. The end result when using an FDM printing process is a 3Dobject 120 having a data matrix (such as 600 of FIG. 6A) embedded into asurface structure thereof. The end result when using a grid-based areatype of printing process is a 3D object (such as 702 of FIG. 7) having adata matrix (such as 706 of FIG. 7) embedded into a surface structurethereof. As previously described, the embodiments described herein relyon the light reflective properties of differently shaped symbols formedby the material in the embedded data matrix to recover the data in amanner that it is relatively easy using conventional 2D image processingtechnology.

The method 200 starts at step 202 and proceeds to step 204, where themethod 200 acquires data, such as barcode data, to be included asinformation with a 3-D object to be printed. The data can be acquired byany number of ways, such as, it can be included as a data file(metadata) along with the STL file of the 3D object. Alternatively, thedata could be input manually by a user of the computer 102 duringpreparation for printing of the 3D object. One example of such datacould be a name or title for the 3D object. Another example, used in thefollowing exemplary description, is a URL, such as “www.google.com”.Alternatively, the URL can point to a different internet site. In thepresent URL example, the data can be divided into 14 bytes of data, suchas“|w|w|w|.|g|o|o|g|l|e|.|c|o|m|”.

The method 200 proceeds to step 206, where the method 200 encodes thedata into code words made up of symbols, using, for example, a subset ofISO/IEC 646 encoding (i.e., a 7-bit coded character set for informationinterchange). One example of such code words created using symbols (suchas English letters and Arabic numerals), described in further detailbelow, is the following hexadecimal sequence: 0xe7, 0x3a, 0x38, 0xce,0x63, 0xb0, 0x7f, 0x1c (note, in this example “Ox” is simply a placeholder, and the next two symbols are used for data representation).Hexadecimal data representation uses sixteen distinct symbols, in theillustrated example symbols 0-9 to represent values zero to nine, andsymbols a, b, c, d, e, f to represent values ten to fifteen. The limitedcharacter subset of ISO/IEC 646 encoding is supported by nearly allcomputer systems available today. As previously described, encodingre-maps symbols in accordance with a pre-defined mapping, so as tofacilitate reliable machine recovery of the data being conveyed (in theface of durability and recovery issues), or to increase the amount ofinformation conveyed or to obscure the presentation of the data beingconveyed, by the code. In order to increase the durability andreliability of the data encoded by the symbols arranged into ahexadecimal sequence, error correction may be employed as part of thecoding process. One suitable known error correction coding technique isReed-Solomon Error Correction Coding (RS-ECC). As is standard practicein RS-ECC, the method 200 estimates the percentage of space needed toincorporate the extra code symbols in accordance with the RS-ECCtechnique, by determining the number of code words which may contain anerror or be concealed without it being detrimental to data recovery whenscanning and reading the symbols. Reed-Solomon codes are especiallywell-suited to applications where recovery errors occur in a pluralityof adjacent portions of a sequence, that is, in bursts. Reed-Solomoncoding calculates complementary codes and add-in codes during thecreation of the symbols. Reed-Solomon decoding regenerates the originalcoded data by recalculating the data from the complementary codes andadd-in codes. The recalculation regenerates the original data bylocating errors that occurred at the time of image scanning recovery ofthe code. Such errors may have been the result of 3D printing problems,specular reflection during the image scanning recovery of the code ordegradation of the surface and volume of the 3D object in which the 3Dcode is embedded.

Continuing with the above example, before the 14 bytes of data (that isthe information: “|w|w|w|.|g|o|o|g|l|e|.|c|o|m|”.) can be converted intoa barcode for association with a 3D object, additional data must prefixthe data. A first prefix is used to indicate the packing method used forthe data. For example, to indicate the known PDF417 packing method, the14 bytes of data becomes the following 15 bytes:“|xE7|w|w|w|.|g|o|o|g|l|e|.|c|o|m|”. A second prefix is used to indicatethe length of the data, that is 14 bytes, thereby causing the data tonow comprise the following 16 bytes:“|xE7|x0C|w|w|w|.|g|o|o|g|l|e|.|c|o|m|”. Although disclosure of furtherdetails of the optional RS-ECC coding is beyond the scope of thisspecification, it is noted that 14 bytes of ECC CodePoints are neededfor RS-ECC of 16 bytes of input data, as calculated by Galois FieldMultiplication with a field generator polynomial. As well known, the ECCCodePoints can be considered as extra points in a coded sequence, sothat if only a part of the coded sequence is recovered, the fullsequence can still be regenerated using only the recovered codes.

Thus, continuing with the above example, 14+16=30 code words need to beincluded in the 3D barcode in order for the 3D barcode to incorporateReed Solomon error correction techniques for the“|w|w|w|.|g|o|o|g|l|e|.|c|o|m|” data.

Given a grid size of M×N (columns and rows) in which to print the 30code words of the 3D barcode, with K distinct values per symbol to beprinted, establishes that:

Number of bits stored=M×N×log₂(K)

Number of codes stored (n)=⅛×M×N×log₂(K)

For FDM symbology with K=4 (corresponding to the four arrangements ofparallel lines shown in FIG. 6A or four arrangements of solid triangularobjects shown in FIG. 7, described below), we can calculate the requiredsize of the data matrix as:

${\# \mspace{14mu} {of}\mspace{14mu} {codes}\mspace{14mu} {stored}\mspace{14mu} (n)} = \frac{M \times N}{4}$

-   -   For M=16, N=8, the number of codes that can be stored is 32        codes, which will accommodate the above-noted requirement of 30        codes.    -   The actual number of rows (N) in the symbol matrix (so as to        include one row each for an Orientation Bar and a Clock track,        respectively, described below in conjunction with FIG. 6A) is        N+2=10.    -   The actual number of columns in the symbol matrix (so as to        include one column each for an Orientation Bar and a Clock        track, respectively, described below in conjunction with FIG.        6A) is M+2=18.

Furthermore, in accordance with the optional Reed-Solomon error coding,the method 200 splits the n codes into (k) data codes and (n−k) errorcodes. With increasing value of n, the ratio of k/n will increase. Thiswill allow correction up to (n−k)/2 erroneous symbols and (n−k)erasures. Thus, in the above example where M=16, N=8 and K=4, if thenumber of detected errors is does not exceed 7 or the number ofunreadable symbols does not exceed 14, the original data can berecovered.

Continuing with the above example, the following hexadecimal sequence ofcode words is representative of the encoded data and ECC code wordswhich collectively express“|xE7|x0C|w|w|w|.|g|o|o|g|l|e|.|c|o|m|”:

-   -   0xe7, 0x3a, 0x38, 0xce, 0x63, 0xb0, 0x7f, 0x1c,    -   0xb2, 0x3f, 0xda, 0x69, 0xc7, 0x92, 0x33, 0xc7,    -   0x81, 0x1c, 0x11, 0xfe, 0xb8, 0x7d, 0x67, 0x88,    -   0x80, 0xa5, 0xc7, 0xeb, 0xdb, 0xc4, 0x0c, 0xcb

A matrix of 10 rows by 18 columns is created having therein 2 rows and 2columns for defining the orientation and bounding area for the symbolmatrix, thereby leaving 8 rows by 16 columns for symbol placement, whichsymbols will be representative of the above-shown coded data sequence.Since only four symbols, representative of the numbers 0, 1, 2 and 3,are being used in the symbol matrix, the above hexadecimal codedsequence must be converted to a sequence able to be represented by base4code words. A workflow for calculating the base4 code words to representthe byte sequence of the 32 hexadecimal code words shown above, is afollows:

-   -   Set NUM_COLUMNS=16    -   Set NUM_ROWS=8        -   For rowIndex in 2 to (MAX_ROWS−1)        -   For column Index in 2 to (MAX_COL−1)            -   Set DataIndex=((rowIndex−2)*NUM_COLUMNS+(column                Index−2))*2+1            -   Matrix                [rowIndex][columnIndex]·bit[0]=data[DataIndex/8]·bit[DataIndex                % 8]            -   Set DataIndex=DataIndex+1            -   Matrix                [rowIndex][columnIndex]·bit[1]=data[DataIndex/8]·bit[DataIndex                % 8]    -   End

As previously noted, the details for coding of the data to be conveyedby the barcode are incidental and optional to the embodiments describedherein, and thus a detailed description of such coding is deemed notnecessary for understanding the present invention. However the aboveworkflow, in simple terms, picks the required number of bits from aninput buffer stream of the hexadecimal data, and populates the symbolmatrix in row major format. For example the first hexadecimal byte 0xe7(where Ox is a place holder and the actual data is “e7”) is equivalentto: (11−10−01−11) base 2=(3 2 1 3)base 4. We can consider that the aboveworkflow converts the base256 (byte) input stream to a base4 outputstream which is used to populate the symbol matrix. The conversion isdone in little endian format (smaller end comes first in the matrix).The method proceeds to step 208.

At step 208 the method 200 determines printing instruction modificationsthat are required to be applied to a 3D printer using a particularprinting technology in order to print the 3D object. Thus, the method200 develops as an electronic file a set of instructions for printingthe 3D object. Upon application of the printing instruction file to a 3Dprinter, the coded data becomes stored as a matrix of symbols (such as620-626 of FIG. 6A), so as to comprise structure that is embedded intothe surface of the 3D object, upon printing. Step 208 is optimized inaccordance with method 300 of FIG. 3 and method 400 of FIG. 4,subsequently described, to achieve accurate symbol printing for aparticular printing technology to be used to print the 3D object. Method300 is optimized for a linear extrusion/tracing type of printingprocess, such as used by an FDM printer, while method 400 is optimizedfor a grid-based layered area type of printing process, such as used bya DLP, PDP, JLP or SLA printer.

The method 200 proceeds to step 210, where the method 200 provides theelectronic file of modifications (set of instructions) determined bystep 208 as layer by layer instructions for application to a 3D printer.Such instructions, when applied to a 3D printer, cause the 3D printer toprint the 3D object, layer by layer, such that the symbol matrix ofcoded data becomes an embedded structural volume (such as matrix 706depicted in FIG. 7), into the surface of the 3D object. The method 200then proceeds to step 212 and ends.

FIG. 3 depicts a flow diagram of a method 300 for storing data embeddedinto the surface structure of a 3D printed object as performed by thedata encoding processor 114 and symbol embedding module 116 of FIG. 1for a first type of printing process, according to one or moreembodiments. More specifically, method 300 embodies steps 208 and 210 ofmethod 200 for an FDM type of printer. Method 300 starts at step 302 andproceeds to step 304.

At step 304, the method 300 initially processes an electronic filerepresentative of the 3D object to be printed, in a manner similar tothat provided by conventional 3-D printing software. That is, the method300 develops and then slices a voxelized version a 3-D object, such as702 of FIG. 7, into layers. As well known, 3-D printing software slicesthe STL file representative of the 3-D object into layers, which layersare then traced out by an extrusion head controlled by G-codes. G-codesare a set of codes that are standard in the industry for controllingmovement of a cutting tool or print head. In the context of a 3Dprinter, the G-codes are applied as an electronic file to control thetracing pattern of the extrusion head, the speed of the extrusion headand whether or not the head is extruding material or not. The method 300analyzes the STL file, in a manner well known to those of ordinary skillin the art, to develop the G-codes that are applied as an electronicfile to control the extrusion head for controllable printing of the 3Dobject.

The method 300 proceeds to step 306, where the method 300 determineslayer by layer modifications to the G-codes, so that upon printing ofthe 3D object, the modified G-codes cause the printer to embed a symbolmatrix such as matrix 600 of FIG. 6A, into the structure of the 3Dobject. The method 300 analyzes the STL file to preferably find a flatsurface on the 3D object that has at least the area of the symbolmatrix. Specifically, the flat surface on the 3D object must be at leastof a size to include the determined number of code words in the symbolmatrix. For example, the size needed for the symbol matrix to hold thecode as determined by step 206 above, may be 5 cm by 2.5 cm, by ½ cmdeep. Thus, in this example, a flat surface on the 3D object must befound that is at least 5 cm by 2.5 cm or larger. The method 300 uses thepreviously developed printing instructions to determine modifications tobe applied to the G-code. Step 306 applies the modifications to theG-code for each layer so as to develop a modified set of G-codes whichwhen applied to an FDM printer, would cause the FDM printer to print the3D object, layer by layer, such that the symbol matrix of coded databecomes an embedded structural volume (such as matrix 600 depicted inFIG. 6A), into the surface of the 3D object. Method 300 then proceeds tostep 308.

At step 308 the method 300 provides the modifications determined by step306 as layer by layer instructions for application to an FDM type of 3Dprinter. Such instructions, when applied to an FDM type of 3D printer,cause the 3D printer to print the 3D object, layer by layer, with thesymbol matrix embedded therein. The method 300 then proceeds to step 310and ends.

FIG. 4 depicts a flow diagram of a method 400 for storing data embeddedinto the surface of a 3D printed object as performed by the dataencoding processor 114 and symbol embedding module 116 of FIG. 1 for asecond type of printing process, according to one or more embodiments.More specifically, a second type of printing process may be of theSLS/DLP/PDP/SLA type. Method 400 constructs a symbol grid, subtracts avolume from the 3D object that corresponds to the volume of the symbolgrid, performs a Boolean union operation that embeds the volume of thesymbol grid with the volume subtracted, and then develops printinginstructions that cause the layer by layer printing of the 3D object ina manner such that the symbol grid becomes an embedded structural volumetherein. The method 400 starts at step 402 and proceeds to step 404.

At step 404, the method 400 develops a symbol matrix such as matrix 706of FIG. 7, so as to represent the coded data 118. Details for theconstruction of a suitable symbol matrix is described below inconjunction with the description of FIG. 6B and FIG. 7. Method 400 thenproceeds to step 406.

Step 406 can be performed before, after or simultaneously with step 404.At step 406, the method 400 receives and processes a STL filerepresentative of the surface of a 3-D object to be printed using wellknown 3-D printing software, into a voxel grid representation of the 3Dobject. That is, the STL file is converted into solid representationcomposed of voxels. The method 400 analyzes the voxel gridrepresentation of the 3D object to preferably find a flat surface on the3D object that has at least the volume need to form the symbol matrix.Specifically, the flat surface on the 3D object must be at least of thesize of the symbol matrix and the depth of the symbol matrix. Forexample, the size needed for the symbol matrix to hold the code may be 5cm by 2.5 cm by ½ cm deep. Thus, in this example, a flat surface on the3D object must be found that is at least 5 cm by 2.5 cm or larger. Themethod 400 subtracts from a flat surface found in the voxel gridrepresentation of the 3D object, preferably on a bottom surface of the3D object, a cuboid volume that extends into the 3D object and whichcorresponds to the volume required to print the symbol matrix. Themethod 400 then proceeds to step 408.

At step 408, the method 400 performs a Boolean union operation thatembeds the volume of the symbol matrix with the cuboid volume removedfrom the voxel grid representation of the 3D object by operation of step406. Thus, the coded print matrix becomes embedded in the bottom surfaceof the voxel grid representation of the 3D object. Method 400 thenproceeds to step 410.

At step 410, the method 400 provides the voxel grid representation ofthe 3D object having the coded symbol matrix embedded therein, to a DLP,PDP or SLA type 3D printer, for printing the 3D object. The method 400proceeds to step 412 and ends.

FIG. 5 is a flow diagram of a method 500 for retrieving coded dataprinted in a surface of a 3D object as performed by the image processingmodule 124 and the data decoding and error correction module 126 of FIG.1, according to one or more embodiments. The method 500 starts at step502 and proceeds to step 504. At step 504, the method 500 receives animage, for example a photograph of the surface of a 3D object having acoded symbol matrix embedded therein. If it is not known in which partof the 3D object the coded symbol matrix is embedded, multiple imagescan be received of the 3-D object from various angles in order that allimages can be analyzed using the following method so as to recover thecoded data. Typically, however it is known where the coded data isembedded, such as on a bottom surface 708 of the 3-D object 702 of FIG.7. Thus, at step 504 a photograph taken, for example, using a cell phonecamera, is received as an image which has captured the coded symbols.

The method 500 proceeds to step 506, where the method 500 usesconventional edge detection, previously described, to enhance the imageso as to find the orientation bars and clock tracks. As noted above, theorientation bars and clock tracks include a predefined pattern ofsymbols therein that identify the location (orientation and extent) ofthe bounding area of the symbol matrix and the grid size that dividesthe bounding area into the sub-regions, which sub-regions identify thematrix portions that include respective ones of the coded symbols.Further description of the orientation bars and clock tracks is providedbelow in conjunction with FIG. 6A.

The method 500 proceeds to step 508, where the method 500 analyzes theedge detected image in each sub-region of the bounding area usingstandard image recognition techniques, such as pattern matching, inorder to recover the symbols representative of the coded data. As notedabove, the symbols may be four unique image patterns, such as fourdifferent spatial orientations of parallel lines or four differentspatial orientations of a given geometric shape, where each differentspatial orientation is representative of a value of 0, 1, 2 or 3, so asto provide 2 bits of information for each symbol.

The method 500 proceeds to step 510, where the method 500 decodes therecovered symbols using standard decoding techniques, such as a look uptable. If Reed-Solomon error correction coding was applied, therecovered symbols include encoded data and ECC code words. The method500 then uses Reed-Solomon error correction decoding in order todetermine the original encoded data by recalculating the recovered data.The original data is determined from complementary codes and the ECCcode words added during the encoding process, described above withrespect to step 206 of method 200. A recalculation of the recovered dataregenerates the original data by locating any errors. Such errors may bethe result of problems during 3-D printing, specular reflection duringthe image generation process or degradation of the coded symbolsembedded in the printed volume. Although the details of Reed-Solomondecoding are beyond the scope of this disclosure and providing themherein would unnecessarily obscure the disclosure, those skilled in thisart will recognize that the standard and well know Peterson Decoder canbe used to decode the recovered code words into the original data aslong as, in the above example where M=16, N=8 and K=4, the number ofdetected errors is does not exceed 7 or the number of unreadable symbolsdoes not exceed 14.

The method 500 proceeds to step 512, where the method 500 provides theregenerated original data, for example to a display, where a user canview the information conveyed by the data. The method 500 then proceedsto step 514 and ends.

FIGS. 6A and 6B together illustrate construction of a symbol matrix 600as performed by the symbol embedding module 116 of FIG. 1, according toone or more embodiments. In the FIG. 6A embodiment, a symbology 601 isused which is optimized for FDM type printing. In symbology 601 fourdifferent arrangements of parallel lines (or ridges) are representativeof easily identified symbols. A symbol comprising an arrangement of leftto right downward diagonal parallel lines corresponds to a numerical 0.A symbol comprising an arrangement of right to left downward diagonalparallel lines corresponds to a numerical 1. A symbol comprising anarrangement of horizontal parallel lines corresponds to a numerical 2. Asymbol comprising an arrangement of vertical parallel lines correspondsto a numerical 3. In the FIG. 6B embodiment, an alternative symbology isshown optimized for non-FDM printers, where each of the symbols can berepresented by a predetermined orientation angle of a slanted surface,such as slanted surface 630, as described in greater detail below inconjunction with the description of FIG. 7.

As previously described, the size needed for the symbol matrix to holdthe code (as determined by step 206 of method 200), may be 10 rows by 18columns. Based on the size of the flat surface on the 3D mode, thismatrix may be determined to easily fit into an area of 5 cm by 2.5 cm by½ cm deep. In order to advantageously provide for simple imagerecognition techniques to be used to easily identify the orientation,shape and size of the symbol matrix 600, L-shaped orientation bars 602and 604 are provided. Orientation bars 602 and 604 are formed into anL-shaped bound defining a square or rectangular shape. Each bar includesa predefined arrangement of symbols which enhance their detection whenthe aforementioned image processing edge detection techniques are used.In the FIG. 6A illustrated embodiment an arrangement of three parallellines 603 is used in orientation bar 602 and an arrangement of threeparallel lines 605 is used in orientation bar 604. The extent of theparallel lines 603 and 605 indicates the x-y size (area) of the matrix600.

Opposing L-shaped sides 606 and 608 of the L-shaped bound are clocktrack bars, and include an arrangement of parallel lines, such as 610and 612, arranged in a repeating pattern as symbols along their length.The length of each symbol along each of the opposing L-shaped sidescorresponds to a unit of measurement. In the illustrated embodiment,adjacent symbols 610 and 612 each identify a length of respectivesub-regions 614 and 616. The length of additional sub-regions alongsides 606 and 608 are identified by additional symbols formed adjacentto symbols 610 and 612, forming a repeating measurement pattern alongsides 606 and 608, hence the term “clock track” for the sides 606 and608. Although only a few of symbols 610 and 612 are shown so as to notobscure the drawing, in practice such symbols are repeated along thefull length of sides 606 and 608 so as to identify the length ofsub-regions within the symbol matrix 600. Thus, when the identifiedlengths of the sub-regions are extended in a parallel pattern across thex-y area of the bounded area including symbol matrix 600, they dividethe bounded area into an identified grid of sub-regions 618. Within eachsub-region 618 a symbol is printed using the material and manufacturingprocess used to print the 3-D object. For an FDM type printer, in oneembodiment noted above, the symbols used in the symbol matrix arerespective ones of four possible arrangements of parallel lines 601shown in FIG. 6A. For non-FDM type printers, in another embodiment, thesymbols used in the symbol matrix are respective ones of four possibleangles of a slanted surface 630 shown in FIG. 6B.

As noted above, in the FIG. 6A embodiment the symbols comprise fourdifferent arrangements of parallel lines, and more specifically, a rightto left arrangement 620, a left to right arrangement 622, a horizontalarrangement 624 and a vertical arrangement 626. Although, the FDM printhead cannot represent data at each layer as compactly as a DLP/PDP/SLAtype printer, the extrusion head has very high precision of motion whiletracing parallel lines. Thus, each one arrangement of the parallel linesdefines one symbol included in the symbol matrix 600. Each symbol cancreate with high precision two bits of data by using the 4 simplepatterns 620-626 of parallel lines.

The alternative embodiment illustrated by FIG. 6B is optimized for aDLP/PDP/SLA type printer, where the material used to form the 3D objectis arranged so as to form a slanted surface within each sub-region 618of the matrix 600. The slanted surface acts as a symbol. In oneembodiment, the symbol in each sub-region has a given orientation of theslant angle of a plurality of possible orientations. Each symbol can bearranged along the orientation bars, clock track bars and in thesub-regions 618. A slanted surface is particularly well-suited forforming a symbol for image recognition, since depending on anorientation angle of the slanted surface 630, a different amount oflight will be reflected from the material forming the surface 630, andthereby vary the brightness of that sub-region in the matrix 600. In theillustrated embodiment the symbology includes four solid right isoscelestriangles 632, 634, 636 and 638, each one orientated so as to be rotated90 degrees around a central vertical axis from one another. Eachparticular orientation of the four triangles is used to convey arespective one of four numeric values, such as 0, 1, 2 and 3. Thedifferent orientations of the slanted surface 630 for each of triangles632, 634, 636 and 638 causes a different level of reflectivity in thatrespective portion of the sub-region of the matrix. Such sub-regions ofdifferent reflectivity are easily detected by the image processingmodule 124 of FIG. 1, thereby providing for relatively easy symbolrecognition and data recovery using the previously described method 500.

Although in the illustrated embodiments the data matrix is embedded intoa flat surface of the 3D object, non-flat surfaces can also be used,although they would not be optimal. For example if a curved surface wereused, geometric transformations would be required to adjust a recovered2D image of the data codes in order to obtain the original geometry ofthe printed symbols. As well known to those skilled in processinggeometric images, such transformation may comprise rotation,translation, scaling, mapping, etc., collectively more generally knownas affine transformations, which transfer shapes from one type ofEuclidean space to another.

The disclosed 3D printed data store technology has significantadvantages over 2D barcodes as specified above in terms of data storagecapacity, durability and security (protection from malicious intent).

Steganography advantages: 2D pasted barcodes are intrusive. They hidethe content and surface underneath. A 3D printed data storage for FDMprinters can be added into the top layers of any flat surface of a 3Dobject with minimum perceptible change to the overall structure of the3D object. The only change required for a FDM based 3D print would be inthe sliced G-code used to form the top layers.

Data storage capacity advantages: Due to the fact that the embodimentsdescribed herein can use a majority of the 3D object surface for storingdata (such as the entire bottom of the base of the object, a much largerregion in which to include the data is provided, as compared with anintrusive 2D barcode label. In addition, due to the high accuracy of FDMprinter's ability to print parallel lines, a small enough symbol sizecan be easily printed which will allow a very high density of datastorage.

Durability advantages: Since the 3D data will be stored as part of thesurface structure of the 3D object itself during the 3D printingprocess, instead of being pasted on to the 3D object later on, the datawill not be able to be removed as easily as a pasted label barcode,which can be easily removed or destroyed when splashed with a liquid. Inaddition, over an entire lifetime of wear and tear on the 3D object, the3D object typically suffers minimal loss of volume below its surface,thereby ensuring durability of 3D data which is embedded into the 3Dobject. Even furthermore, by use of Reed Solomon Error Correction codetechniques, recovery from (n−k)/2 erroneous symbols and (n−k) erasuresis possible. For typical ratios of n and k, this will allow for recoveryfrom up to 20% erroneously recovered symbols and 40% erasures over alifetime of surface degeneration.

Security (protection from malicious intent) advantages: Since the codeddata is stored as an embedded volume, it cannot be easilyremoved/scraped/changed without noticeable change in surface geometryand texture of the 3D object, thereby allowing for storing data on thesurface, such as a hidden watermark. A pasted barcode can be removedmuch more easily.

The present embodiments can be used for authoring 3D objects which havedatacodes embedded in their surface for storing data, such as foridentifying a part number and storing other data such as:

-   -   Batch number    -   URL    -   Watermark    -   Production date    -   Package date    -   Serial number    -   Weight (kgs/pounds)    -   Price    -   UPS Code

The embodiments of the present invention may be embodied as methods,apparatus, electronic devices, and/or computer program products.Accordingly, the embodiments of the present invention may be embodied inhardware and/or in software (including firmware, resident software,micro-code, etc.), which may be generally referred to herein as a“circuit” or “module”. Furthermore, the present invention may take theform of a computer program product on a computer-usable orcomputer-readable storage medium having computer-usable orcomputer-readable program code embodied in the medium for use by or inconnection with an instruction execution system. In the context of thisdocument, a computer-usable or computer-readable medium may be anymedium that can contain, store, communicate, propagate, or transport theprogram for use by or in connection with the instruction executionsystem, apparatus, or device. These computer program instructions mayalso be stored in a computer-usable or computer-readable memory that maydirect a computer or other programmable data processing apparatus tofunction in a particular manner, such that the instructions stored inthe computer usable or computer-readable memory produce an article ofmanufacture including instructions that implement the function specifiedin the flowchart and/or block diagram block or blocks.

The computer-usable or computer-readable medium may be, for example butnot limited to, an electronic, magnetic, optical, electromagnetic,infrared, or semiconductor system, apparatus, device, or propagationmedium. More specific examples (a non-exhaustive list) of thecomputer-readable medium include the following: hard disks, opticalstorage devices, a transmission media such as those supporting theInternet or an intranet, magnetic storage devices, an electricalconnection having one or more wires, a portable computer diskette, arandom access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM or Flash memory), an optical fiber,and a compact disc read-only memory (CD-ROM).

Computer program code for carrying out operations of the presentinvention may be written in an object oriented programming language,such as JAVA®, Smalltalk or C++, and the like. However, the computerprogram code for carrying out operations of the present invention mayalso be written in conventional procedural programming languages, suchas the “C” programming language and/or any other lower level assemblerlanguages. It will be further appreciated that the functionality of anyor all of the program modules may also be implemented using discretehardware components, one or more Application Specific IntegratedCircuits (ASICs), or programmed Digital Signal Processors ormicrocontrollers.

The foregoing description, for purpose of explanation, has beendescribed with reference to specific embodiments. However, theillustrative discussions above are not intended to be exhaustive or tolimit the invention to the precise forms disclosed. Many modificationsand variations are possible in view of the above teachings. Theembodiments were chosen and described in order to best explain theprinciples of the present disclosure and its practical applications, tothereby enable others skilled in the art to best utilize the inventionand various embodiments with various modifications as may be suited tothe particular use contemplated.

The methods described herein may be implemented in software, hardware,or a combination thereof, in different embodiments. In addition, theorder of methods may be changed, and various elements may be added,reordered, combined, omitted, modified, etc. All examples describedherein are presented in a non-limiting manner. Various modifications andchanges may be made as would be obvious to a person skilled in the arthaving benefit of this disclosure. Realizations in accordance withembodiments have been described in the context of particularembodiments. These embodiments are meant to be illustrative and notlimiting. Many variations, modifications, additions, and improvementsare possible. Accordingly, plural instances may be provided forcomponents described herein as a single instance. Boundaries betweenvarious components, operations and data stores are somewhat arbitrary,and particular operations are illustrated in the context of specificillustrative configurations.

Other allocations of functionality are envisioned and may fall withinthe scope of claims that follow. Finally, structures and functionalitypresented as discrete components in the example configurations may beimplemented as a combined structure or component. These and othervariations, modifications, additions, and improvements may fall withinthe scope of embodiments as defined in the claims that follow.

While the foregoing is directed to embodiments of the present invention,other and further embodiments of the invention may be devised withoutdeparting from the basic scope thereof, and the scope thereof isdetermined by the claims that follow.

1. A computer implemented method comprising: receiving a firstelectronic file representative of a 3D object to be printed; receivingdata to be presented in association with the 3D object to be printed;developing a 3D symbol matrix comprising a plurality of symbolsrepresentative of the data; and developing a second electronic file tobe used for printing the 3D object, where the second electronic fileembeds the 3D symbol matrix into the 3D object.
 2. The method of claim1, further comprising determining a selected symbology of a plurality ofsymbologies to be used for making the symbol matrix, which selectedsymbology is determined based on the type of printing process to be usedto print the 3D object.
 3. The method of claim 1, where developing thesecond electronic file comprises developing G-code printing instructionswhich, when applied to a Fused Deposition Model (FDM) printer, causesprinting of the 3D object with the 3D symbol matrix embedded therein. 4.The method of claim 1, where developing the second electronic filecomprises developing sequential area layers which, when applied to a 3Dprinter, causes printing of the 3D object with the 3D symbol matrixembedded therein.
 5. The method of claim 1, where developing the 3Dsymbol matrix comprises dividing the 3D symbol matrix into a pluralityof sub-regions, each sub-region including a symbol representative of aportion of the data.
 6. The method of claim 5, further comprisingstructuring the symbol in each sub-region to cause a predeterminedpattern of light reflectivity by the material used to print the 3Dobject in the area of the sub-region.
 7. The method of claim 5, furthercomprising structuring the symbol in each sub-region to be onearrangement of parallel lines of a plurality of possible arrangements ofparallel lines.
 8. The method of claim 5, further comprising structuringthe symbol in each sub-region to be one orientation of a slanted surfaceof a plurality of possible orientations of a slanted surface.
 9. Amethod to retrieve data embedded as structure in the surface of a 3Dobject, comprising: obtaining a 2D image of at least a portion of a 3Dobject having symbols embedded as structure in the surface thereof;processing the 2D image to identify an orientation and dimension of anarea of the 3D object that includes the embedded symbols; processing the2D image to identify sub-regions of the identified area; processing the2D image to detect symbols in the identified sub-regions; and processingthe detected symbols by decoding so as to retrieve data from thesymbols.
 10. The method of claim 9, where processing the 2D image todetect symbols in the sub-regions detects for each symbol onearrangement of parallel lines of a plurality of possible arrangements ofparallel lines.
 11. The method of claim 9, where processing the 2D imageto detect symbols in the sub-regions detects for each symbol oneorientation of a slanted surface of a plurality of possible orientationsof a slanted surface.
 12. The method of claim 9, where processing the 2Dimage to identify an orientation and dimension of an area, identifies afirst pair of joined L-shaped bars as a perpendicular arrangement of twogroups of parallel lines, the extent of each group of parallel linesdefining a length dimension of the area.
 13. The method of claim 12,where processing the 2D image to identify sub-regions of the area,identifies a second pair of L-shaped bars as a repeating pattern ofsymbols, a dimension of each symbol in the repeating pattern defining adimension of a sub-region of the area.
 14. The method of claim 13, wherethe repeating pattern of symbols comprises two adjacent symbols, eachsymbol having a different one orientation of a plurality of possibleorientations.
 15. A printed 3D object comprising: a surface havingstructurally embedded therein during printing, a volume corresponding toa 3D symbol matrix comprising an arrangement of symbols representativeof data.
 16. The 3D object of claim 15, wherein each symbol comprisesone orientation of a plurality of possible orientations of a either aslanted surface or a plurality of parallel lines.
 17. The 3D object ofclaim 15, wherein: the 3D symbol matrix presents at a surface of the 3Dobject, an area identifiable by indicia that define an orientation anddimension of an area of the 3D symbol matrix embedded in the 3D object;and indicia that define sub-regions of the area, where the symbolsrepresentative of the data are printed in the sub-regions.
 18. The 3Dobject of claim 17, wherein the symbol in each sub-region causes apredetermined pattern of light reflectivity by the material of the 3Dobject in the area of the sub-region.
 19. The 3D object of claim 17,wherein the indicia that define an orientation and dimension comprise apair of joined L-shaped bars, each bar comprising a uniform arrangementof adjacently positioned symbols, the extent adjacently positionedsymbols defining a length dimension of the area.
 20. The 3D object ofclaim 17, wherein the indicia that define sub-regions of the areacomprise a pair of L-shaped bars, each bar comprising a repeatingpattern of symbols, a dimension of each symbol in the repeating patterndefining a dimension of a sub-region of the area.